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Abstract 

Linear  quadratic  Gaussian  control  is  a  technique  that  uses  Kalman  filtering  to  estimate  a  state 
vector  used  for  input  into  a  control  calculation.  A  control  correction  is  calculated  by  minimizing  a 
quadratic  cost  function  that  is  dependent  on  both  the  state  vector  and  the  control  amount.  Different 
penalties,  chosen  by  the  designer,  are  assessed  by  the  controller  as  the  state  vector  and  control  amount 
vary  from  given  optimal  values.  With  this  feature  controllers  can  be  designed  to  force  the  phase  and 
frequency  differences  between  two  standards  to  zero  either  more  or  less  aggressively  depending  on  the 
application.  Data  will  be  used  to  show  how  using  different  parameters  in  the  cost  function  analysis 
affects  the  steering  and  the  stability  of  the  frequency  standards. 


INTRODUCTION 

The  steering  of  frequency  standards  (atomic  clocks)  is  a  very  important  procedure  in  the  timing 
community.  Steering  is  used  to  synchronize  remote  clocks  using  very  accurate  time  transfer 
methods  such  as  two-way  time  transfer  and  the  Global  Positioning  System.  Also  in  time  scale 
applications  a  standard  is  steered  to  a  paper,  or  calculated,  clock  in  order  to  give  the  time 
scale  a  physically  realizable  output.  This  paper  will  discuss  how  the  linear  quadratic  Gaussian 
(LQG)  technique  applies  to  the  designing  of  control  systems  to  steer  frequency  standards. 

In  any  real  world  application,  a  control  system  must  deal  with  some  amount  of  uncertainty, 
whether  it  comes  in  the  form  of  sensor  noise,  process  modeling  error,  or  any  other  noise 
sources.  The  LQG  technique  is  used  for  designing  optimal  control  systems  for  uncertain 
physical  processes.  An  important  feature  of  this  technique  is  that  the  stability  of  the  control 
system  is  assured  if  system  parameters  have  the  properties  of  observability  and  controllability. 
Kalman  filtering  is  used  in  order  to  estimate  the  actual  state  variables  from  measurements 
made  of  the  stochastic  system. 


TWO-STATE  LQG  THEORY  FOR  FREQUENCY  STANDARDS 

In  the  LQG  theoryHA3!  the  state  equation  is  assumed  to  be  given  as  a  linear  function  of  a 
state  vector  and  a  control  vector: 


x(fc  -I-  1)  =  $x(fc)  -f  Bu(fc)  +  w (k), 


(1) 
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where 


x(k)  =  state  vector  = 

xj(k)  is  the  phase  difference,  and  x2(k)  is  the  fractional  frequency  difference,  between 
the  reference  and  the  steered  standard, 

u(fc)  =  control  vector  which  is  a  scalar  in  this  case  corresponding  to  the  fractional 
frequency  change  of  the  synthesizer  controlling  the  steered  standard, 

$  =  transition  matrix  —  q  \  ,  r  is  the  time  interval  between  measurements, 

w(Jfc)  =  white  noise  characterized  by  covariance  Qk, 

5/10T +  2fe_iTJ  +  I^A-ar3  .  ,  .,  ,  ,  .  ,  ,  , 

Qk=  2  jT2  2ft  2r  ’  an<*  t^ie  h  s  are  calculated  from  the  two-sample 

(Allan)  variance  of  the  comparison  between  two  standards 
The  noisy  measurement  z(fc)  is  related  to  the  state  vector  by 


*,(*) 
x2(A0  ’ 


z(fc)  =  Hx(*)  +  v(Jfe)  (2) 

where 

z(i)  =  measurement,  in  our  case  a  scalar  phase  difference, 

H  =  connection  matrix  =  |  1  0  |,  and 

v(fc)  =  white  noise  characterized  by  covariance  Rk  =  measurement  noise. 

The  linear  state  equation  (1)  is  an  approximation  to  the  state  modeling  equation  that  includes 
higher  order  terms.  In  order  to  help  give  the  linear  approximation  validity,  the  control  vector 
u(fc)  is  chosen  such  that  the  quadratic  cost  function 

J  =  S  [x(fc)rWQx(A;)  +  u(fc)rWjiu(fc)]  (3) 

is  minimized. 

W q  and  W n  are  matrices  that  are  chosen  by  the  designer  in  order  to  set  the  relative  penalties 
assessed  to  the  state  vector  estimate  x{k)  and  control  vector  u(ifc)  as  they  vary  from  zero.  In 
general,  if  W n  is  large  compared  to  Wq,  the  penalty  is  large  for  the  system  attempting  to 
drive  the  state  vector  toward  zero  too  rapidly.  Conversely,  if  Wq  is  large  compared  to  Wjj, 
the  system  faces  a  smaller  penalty  for  large  control  effort  and  the  system  is  driven  toward  zero 
more  quickly. 

Due  to  the  noisy  measurement  of  x(&),  we  are  faced  with  a  compound  problem  of  optimal 
control  and  estimation.  A  very  useful  theorem  from  control  theory  known  as  the  separation 
principle  allows  us  to  solve  the  optimal  control  and  the  estimation  problem  independently. 
Kalman  filtering  is  the  technique  used  to  estimate  the  true  state  x(fc)  from  the  noise.  The 
Kalman  filter  is  calculated  as  usual  PI  with  the  exception  that  the  update  estimate  must  now 
include  control  terms: 


x(fc  -f  1)  =  #x(fc)  +  Bu(A)  +  Kp[z(A:  +  1)  —  H(#x(&)  +  Bu(i))] 


(4) 
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where  K,  is  the  Kalman  filter  gain,  x(A)  is  the  state  estimation,  and  B  —  j  for  the  two-state 
model  with  a  frequency  synthesizer  as  the  control  mechanism.  The  optimal  control  for  the 
given  cost  equation  is 


u(Jfc)  =  —Gox(k) 


(5) 


where 


Go  =  (BrKoB  +  WR)-3BrKo* 


(6) 


and  Ko  is  a  solution  to  the  steady  state  Ricatti  equation 

K<j  =  *rKo*  +  WQ  -  #rKoB(BrKoB  +  W„)-aBTKo*.  (7) 

This  gives  us  a  statistically  optimal  control  u(fc)  for  the  given  cost  function  with  the  designer 
specified  parameters  W q  and  Wn.  Now  that  the  control  is  optimized,  we  need  to  be  concerned 
with  the  stability  of  the  control  design.  Stability  is  assured  if  the  pair  (W<j,  $)  are  observable, 
the  pair  (4>,  B)  are  controllable,  the  Kalman  filter  is  stable,  and  the  model  is  reasonably  good 
(see  [2]).  Controllability  is  the  ability  to  steer  the  system  from  an  initial  state  to  another  state 
in  a  finite  amount  of  time,  and  observability  is  the  ability  to  determine  the  state  at  any  time 
from  a  finite  number  of  measurements. 


SIMULATIONS 

Actual  data  measured  from  frequency  standards  at  the  United  States  Naval  Observatory  (USNO) 
were  used  in  the  simulations.  An  LQG  control  was  applied  to  the  data  as  if  one  of  the  standards 
frequency  was  being  adjusted  by  a  frequency  synthesizer.  Thus,  the  only  assumption  in  the 
simulations  is  that  the  synthesizer  works  ideally. 

The  hydrogen  maser  NAV8  was  chosen  to  be  steered  to  the  USNO  MeanW.  The  Mean  is 
a  paper  clock  that  is  calculated  using  an  ensemble  of  hydrogen  masers  and  cesium  frequency 
standards.  Maser  NAV8  has  excellent  short-term  stability,  but  due  to  the  poor  environment 
that  it  was  in  during  the  data  collection,  its  long-term  stability  suffered.  One  of  the  best 
performing  standards  at  USNO  is  maser  NAV4.  Figure  1  shows  the  performance  differences 
between  NAV8  and  the  Mean  versus  NAV4.  As  can  be  seen,  we  face  an  interesting  problem 
of  steering  maser  NAV8  to  the  Mean  in  phase  and  frequency  while  attempting  to  preserve  the 
short-term  stability  of  the  maser  and  gain  the  long-term  stability  of  the  USNO  Mean.  The 
phase  difference  between  NAV8  and  the  Mean  is  given  in  Figure  2.  In  order  to  minimize 
initial  offsets,  a  frequency  offset  was  removed  from  the  data,  and  a  constant  was  subtracted  out 
giving  the  initial  phase  difference  point  to  be  near  zero. 

In  trial  1  we  set  Wn  =  10s  and  Wq  =  |o°01ooi  J>  gives 

Go  =  |  6.50277  x  10~5  .57714  | 

after  solving  equations  (5)  and  (6).  Figure  3  shows  the  phase  difference  between  the  Mean 
and  NAV8  after  steering  NAV8  using  the  above  solution.  The  phase  difference  is  kept  very 
small  with  the  difference  having  a  standard  deviation  of  140  picoseconds. 


259 


In  trial  2  we  set  Wjt=  io12  and  We=  |  o^^.ooi  j  which  gives 
Go  =  }  3.3185  x  10“8  .014976  ) 

after  solving  equations  (5)  and  (6).  Figure  4  shows  the  phase  difference  between  the  Mean 
and  NAV8  after  steering  NAV8  using  the  trial  2  parameters.  The  phase  differences  after  the 
initial  settling  have  a  standard  deviation  of  691  picoseconds. 

A  plot  of  the  two-sample  deviation  of  NAV4  versus  the  steered  NAV8  for  both  trial  1  and  trail 
2  parameters  is  shown  in  Figure  5.  This  plot  shows  that  the  short-term  stability  of  the  maser 
in  trial  1  has  been  perturbed  by  the  fairly  aggressive  steering.  While  for  trial  2,  the  stability 
exhibits  the  short-term  stability  of  the  maser  and  excellent  performance  in  the  long  term. 

Another  application  of  the  LQG  technique  is  the  steering  of  remote  clocks  to  UTC  (USNO) 
via  GPS.  Figure  6  shows  data  obtained  between  a  keyed  GPS  receiver  and  Hewlett-Packard 
HP5071  cesium  standard  #249.  The  initial  data  had  50  nanosecond  phase  and  4.0  x  10-14 
frequency  offsets.  Also  shown  in  Figure  6  is  the  phase  difference  after  a  simulation  run  with 
the  LQG  control  using  the  parameters  of  trial  1.  We  assume  that  there  are  two  remote  clocks 
being  compared  by  a  noisy  GPS  measurement  system.  The  stability  plot  in  Figure  7  shows  how 
the  cesium  performed  during  the  steers  compared  to  a  hydrogen  maser  after  the  initial  settling 
of  the  controlled  system.  The  solid  line  on  the  plot  shows  the  performance  specification  for 
the  5071  cesium.  The  slightly  worse  stability  near  10  hours  is  most  likely  due  to  modelling 
errors  incurred  from  assuming  whiteness  of  the  GPS  data. 

The  parameters  chosen  for  the  LQG  depend  on  the  systems  being  used,  the  desired  outcome, 
and  the  individual  designer.  This  can  be  seen  in  the  differences  between  the  results  in  trial 
1  and  2.  In  trial  1  the  short-term  stability  is  sacrificed  slightly  for  a  tight  control  in  the 
differences  between  the  standards.  The  stability  is  still  good,  but  if  this  does  not  meet  the 
frequency  stability  needs  for  a  system  then  the  parameters  of  trial  2  could  be  used,  ot  any 
other  parameter  set  that  gives  the  desired  results  as  determined  through  simulation. 


EXPERIMENTAL  RESULTS 

One  of  the  great  concerns  in  designing  a  controller  is  whether  or  hot  it  will  be  stable  and 
robust.  This  was  tested  by  offsetting  an  external  synthesizer,  called  an  Auxiliary  Output 
Generator  manufactured  by  Sigma  Thu  Standards  Corporation,  driven  by  maser  NAV2.  The 
phase  offset  made  was  approximately  8  milliseconds  compared  to  maser  NAV4.  Figure  8  shows 
how  the  controller  with  parameters  given  in  trial  1  of  the  simulations  reacted  to  this  phase  step 
that  was  nearly  7  orders  of  magnitude  greater  than  would  be  expected  in  practice.  The  system 
remained  stable  and  brought  the  signals  within  300  picoseconds  in  approximately  6  days. 

Figure  9  shows  experimental  data  of  maser  NAV2  being  steered  to  the  USNO  Master  Clock 
using  an  Auxiliary  Output  Generator  that  received  its  input  from  a  distribution  amplifier  driven 
by  NAV2.  The  several  hundred  picosecond  humps  in  the  data  are  caused  by  temperature 
changes  in  the  testing  lab  where  the  5  MHz  distribution  amplifier  with  a  poor  temperature 
coefficient  resides.  Temperature  control  of  the  lab  was  poor  during  the  installation  of  a  back-up 
air  conditioning  system. 
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CONCLUSION 


The  LOG  design  philosophy  is  a  robust,  statistically  optimal  method  for  steering  frequency 
standards.  Simulations  can  be  run  without  undue  difficulty  in  order  for  the  designer  to 
characterize  how  different  parameters  will  affect  system  responses.  This  technique  could  also 
be  used  to  steer  one  standard  very  tightly  to  another,  thus  creating  an  independent  back-up  that 
is  in  phase  and  on  frequency  with  its  reference.  Testing  is  now  under  way  for  implementing 
the  LQG  technique  to  synchronize  remote  systems  using  the  Global  Positioning  System  and 
two-way  satellite  time  transfer  methods. 
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TWO-SAMPLE  ALLAN  DEVIATION 
NAV4  -  NAV8(STEERED) 
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Questions  and  Answers 

ALBERT  KIRK  (JPL):  I  have  actually  three  questions.  The  first  one,  what  are  the 
temperature  variations  in  your  laboratoiy? 

PAUL  KOPPANG  (USNO):  They  were  approximately  five  to  six  degrees  C. 

ALBERT  KIRK  (JPL):  I  see.  The  second  question  is:  What  is  the  smallest  step  you  can 
use  on  your  synthesizer  to  correct  the  frequency? 

PAUL  KOPPANG  (USNO):  KT19 

ALBERT  KIRK  (JPL):  The  final  question  is:  How  do  you  determine,  or  how  does  your 
system  determine,  the  loop  time  constant  for  each  maser  in  response  to  —  you  know,  to  steer 
the  maser  to  some  average  that  you  mentioned  here? 

PAUL  KOPPANG  (USNO):  That’s  done  by  the  Kalman  filtering;  it  would  steer  to  a  Kalman 
filter  value. 

ALBERT  KIRK  (JPL):  Can  you  select  that,  then,  for  each  particular  maser,  depending  on 
its  characteristics? 

PAUL  KOPPANG  (USNO):  Yes. 
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